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Statistical  Techniques  for  Signal  Processing:  Final  Report 
AFOSR  Grant  90-0050,  11/1/89-10/31/92 

In  this  report  we  summarize  our  activity  during  the  last  three  years  under  the  above 
research  grant.  Some  significant,  original,  and  interesting  results  have  been  obtained  and 
reported  during  this  penod:  these  have  been  listed  as  publications  [  1  }-| 28 1  on  page  7. 

During  this  period,  under  AFOSR  support,  two  Ph.D.  dissertations  (1.2)  and  two 
Master  s  theses  [3,4]  were  completed,  and  a  book  chapter  (5)  was  written.  Eleven  journal 
papers  were  prepared  (including  one  under  review),  and  twelve  conference  papers  were 
presented.  AFOSR  support  during  this  period  provided  salary  for  two  summer  months  for 
the  Principal  Investigator,  and  twelve  months  of  support  for  one  graduate  research 
assistant,  each  year. 

The  investigations  that  we  have  conducted  during  this  research  grant  penod  may  be 
categorized  as  falling  into  two  broad  areas:  these  are  the  areas  of  Nonlinear  and  Robust 
Filters,  and  of  Signal  Detection  in  Non-Gaussian  Noise.  We  describe  below  the  primary 
results  we  have  obtained  in  these  two  areas,  as  well  as  other-new  results  and  activity. 

A.  Nonlinear  and  Robust  Filters:  Order  Statistics  and  Related  Filters, 

Multivariate  Medians,  Analysis,  and  Applications. 

Nonlinear  filtering  based  on  order  statistics  and  related  techniques  (e.g.  median 
filtering)  has  been  of  much  interest  in  statistical  signal  processing.  Our  work  has  been 
recognized  as  having  contributed  some  very  significant  ideas,  analyses,  and  results  in  the 
development  of  this  key  area  of  signal  processing  research  and  applications.  Dunng  this 
grant  period  we  have  continued  to  make  some  significant  advances  on  several  aspects  of 
nonlinear  and  robust  filters,  as  described  below. 

Nonlinear  filters  such  as  the  L-filter  (based  on  linear  combinations  of  order 
statistics)  are  useful  for  robust  edge- preserving  filtering  and  smoothing  in  non-Gaussian 
(impulsive)  noise.  Such  filters  do  not,  however,  allow  control  over  the  frequency  response 
which  is  a  very  desirable  feature  offered  by  non-robust  linear  filters.  The  combination 
filter,  or  C-filter,  is  a  nonlinear  filter  for  real-valued  inputs  (univariate  case)  which 
combines  the  desirable  characteristics  of  linear  finite -impulse  response  and  nonlinear  L- 
filters.  Such  filters  are  of  interest  as  edge-preserving  robust  filters  which  are  capable  also 
of  providing  "frequency  response"  control.  They  allow  frequency-sensitive  functions  such 
as  deconvolution  or  interference  rejection  to  be  effectively  performed  in  the  presence  of 
impulsive  noise  and  without  loss  of  signal  details  or  edges.  Combination  filters  are  based 
on  joint  time-order  and  rank-order  processing  of  input  data.  Our  contributions  and  results 
on  this  useful  filter  structure  have  been  published  in  the  journal  paper  [10].  Given  in  [10] 
are  several  interesting  illustrations  of  the  various  applications  of  C-filters,  in  image 
processing,  channel  equalization,  and  deconvolution.  Another  application  of  C-filters  in 
detection  in  impulsive  environments  has  been  described  in  [23],  The  dissertation  [1] 
contains  further  details  on  C-filters. 

We  also  began  during  this  grant  period  a  new  effort  focused  on  development  of 
multivariate  nonlinear  (order  statistics)  filters.  Our  particular  focus  has  been  on  multivariate 
extensions  of  the  median  and  related  filters,  for  use  in  applications  such  as  multi- spectral 
image  processing  and  narrowband  (complex-valued  data)  signal  processing.  While  the 
univariate  median  filter  is  easy  to  define,  extensions  to  the  multivariate  case  are  not  so 
obvious  because  there  is  no  natural  criterion  for  rank-ordering  in  p-dimensional  Euclidean 
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space  for  p>l.  We  have  made  some  interesting  observations  and  definitions  in  recent 
preliminary  work  [19,  24,  25],  and  are  working  To  further  these  results.  In  particular,  in 
[24]  and  [25]  we  have  defined  a  new-  bivariate  median  (the  radial  median  i  that  appears  to 
enjoy  some  useful  properties  and  justification  as  a  natural  extension  of  the  univariate 
median.  Us  extension  to  higher  dimensions  produces  a  rather  complex  implementation, 
however  and  needs  additional  work.  The  thesis  [4]  contains  some  other  observations  and 
results  on  bivariate  median  filters. 

In  addition  to  this  effort  on  multivariate  nonlinear  filters,  we  are  continuing  to 
obtain  a  better  understanding  of  the  univariate  robust  edge-preserving  properties  of  median- 
type  filters,  by  relating  their  properties  directly  to  the  influence  functions  and  extended 
influence  functions  of  the  associated  underlying  estimators.  An  interesting  application  of 
the  influence  function  and  its  extensions  has  been  made  by  us  in  analyzing  the  edge- 
preserving  and  smoothing  behavior  of  nonlinear  filters  such  as  the  order-statistics  based  L- 
filters  and  related  filter  structures.  While  the  influence  function  is  best  known  for  its  role  in 
characterizing  the  robustness  of  estimators,  the  mathematical  framework  underlying  it  also 
gives  asymptotic  expressions  for  the  moments  of  estimators.  We  have  made  use  of  this, 
and  obtained  the  influence  functions  and  approximate  performance  characterization  of  filters 
such  as  the  modified  trimmed  mean  filter.  For  such  filters  exact  analysis  is  often  quite 
difficult.  We  have  shown  that  the  approximate  asymptotic  results  are  in  good  agreement 
with  results  obtained  from  exact  analysis  tin  simple  cases)  and  simulations.  This  approach 
and  results  have  been  given  in  [22,26]  and  in  a  recent  journal  paper  [  15].  We  are  extending 
this  method  to  characterize  other  filters  with  more  general  structures  in  current  work.  In  a 
recent  paper  [27]  we  have  discussed  how  a  general  adaptive  filter  structure  based  on  order 
statistics  can  be  designed  and  analyzed  for  performance  using  the  influence  function  as  a 
tool.  In  particular,  we  have  examined  the  mean-median  hybrid  filter  (which  adapts 
smoothly  between  averaging  inputs  and  using  their  median,  depending  on  the 
characteristics  of  the  input)  as  an  example  of  this  type  of  application.  Even  such  a  simple 
filter  is  difficult  to  analyze  exactly  for  performance  with  a  non-homogeneous  (edge¬ 
bearing)  signal  input.  We  have  shown  in  a  journal  paper  that  has  been  accepted  for 
publication  [16]  that  the  influence  function  approach  is  very  useful  in  characterizing  the 
actual  performance  of  this  type  of  filter,  and  have  described  its  use  in  the  edge-preserving 
smoothing  of  noisy  images. 

The  use  of  order  statistics  has  been  widely  considered  for  the  purpose  of  constant 
false  alarm  rate  (CFAR)  detection  of  radar  pulses  in  noise  and  clutter.  For  CFAR  detection 
one  has  to  be  able  to  automatically  set  detector  test  parameters  to  achieve  a  fixed  value  for 
the  false-alarm  probability,  because  the  clutter  and  noise  power  is  not  known.  At  the  same 
time,  the  adaptive  parameter  setting  algorithm  has  to  be  able  to  follow  abrupt  changes  in 
clutter  power  levels,  otherwise  excessive  false  alarms  are  generated  at  such  clutter  edges. 
Some  new  ideas  for  CFAR  radar  detection  in  exponentially  distributed  clutter  have  been 
developed  in  this  grant  period.  Adaptive  selection  of  an  order  statistic  based  on  different 
subsets  of  the  data  window  has  been  described  in  [17].  We  have  improved  upon  this 
scheme  further,  and  a  full  paper  on  this  approach  is  being  prepared  for  publication. 
Another  approach  uses  trimmed  means,  with  a  fixed  amount  of  lower  order-statistics 
trimming  but  with  a  variable,  adaptive  trimming  of  the  higher  order-statistics.  We  have 
been  able  to  demonstrate  useful  gains  in  performance  for  such  a  CFAR  scheme,  and  a  full 
paper  on  this  approach  has  been  accepted  for  publication  [  12]. 
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B .  Signal  Detection  in  Non-Gaussian  Noise:  ARE,  Quantization. 

Distributed  Detection,  Nonparametric  Detection 

One  significant  advance  that  we  have  made  in  the  area  of  signal  detection  and  non- 
Gaussian  noise  modelling  followed  from  our  use  of  new  models  for  describing  signals  and 
noise  which  are  not  simply  additive  and  independent,  a  simplifying  restriction  under  which 
most  previous  studies  have  been  made.  Our  multiplicative  and  signal-dependent  noise 
models  allow  new  results  and  insights  to  be  developed  which  are  of  both  theoretical  and 
practical  significance.  Papers  (6)  and  [7]  describe  our  ideas  and  detailed  results  on  new 
statistical  models  for  use  in  ihe  study  of  signal  detection  and  estimation  schemes  in  non- 
Gaussian  noise.  They  allow  observations  to  be  described  as  desired  signals  in  additive  as 
well  as  multiplicative  noise  and  signal  dependent  noise,  and  provide  in  particular  the 
structures  of  locally  optimum  detection  schemes.  These  results  are  interesting  as  additions 
to  the  body  of  previous  theoretical  results  which  had  been  obtained  for  simple  models  only, 
and  they  provide  insights  useful  in  practice  regarding  optimum  processing  schemes  in  the 
presence  of  noise  which  is  not  only  additive. 

The  asymptotic  relative  efficiency  (ARE)  is  a  useful  asymptotic  measure  for 
gauging  the  relative  performance  of  two  detectors.  It  is  particularly  useful  in  comparing 
nonlinear  detectors  operating  under  non-Gaussian  conditions.  The  ARE  provides  an 
appealingly  simple  measure  to  characterize  the  asymptotic  relative  performance  of  two  tests, 
but  the  relative  performance  for  finite  sample  sizes  is  known  to  sometimes  converge  very 
slowly  to  the  ARE  value.  One  pan  of  our  effon  was  devoted  to  enhancing  the  utility  of  the 
asymptotic  relative  efficiency  (ARE)  as  a  measure  of  relative  performance  of  signal 
detectors.  In  our  work  we  have  been  able  to  give  a  characterization  and  development  of  the 
ARE  which  allows  it  to  more  accurately  predict  observed  finite-sample-size  detector  relative 
performances  (such  as  overshoots  in  the  relative  efficiency  above  the  ARE  value).  The 
work  reported  in  [  18]  and  in  the  paper  [8]  focuses  on  the  canonical  problem  of  detection  of 
known  signals  in  additive  noise,  and  provides  analytical  insights  and  simple  enhancements 
of  the  ARE  measure  to  allow  finite  sample  size  relative  performance  to  be  more  accurately 
predicted.  We  expect  that  this  work  can  be  extended  to  other  scenarios. 

We  have  also  obtained  new  results  on  detectors  using  quantized  observations, 
where  our  initial  focus  has  again  been  on  the  additive  noise  model.  The  use  of  quantized 
data  is  necessary  in  many  situations,  and  optimum  quantization  schemes  for  signal 
detection  have  received  much  attention.  We  have  considered  the  practical  case  where  the 
partitioning  of  input  observations  is  constrained  to  be  time-invariant,  even  frr  non¬ 
stationary  signals,  and  have  obtained  new  results  extending  earlier  solutions.  Tnis  work 
also  made  use  of  the  new  noise  models  we  described  in  [6]  and  [7].  One  interesting 
observation  was  that  even  for  the  classical  case  of  Gaussian  signals  in  Gaussian  noise,  the 
locally  optimum  quantization  of  data  is  not  necessarily  symmetric.  These  new  results  have 
been  published  in  [9,  20]. 

The  detection  of  signals  in  noise  using  observations  collected  over  a  number  of 
sensors  distributed  over  a  spatial  field,  and  which  do  not  communicate  directly  with  one 
another,  has  formed  one  main  focus  of  our  work  on  signal  detection  theory.  We  have  been 
working  towards  the  goal  of  extending  the  well-known  developments  in  single-sensor  (or 
centralized)  detection  theory  to  the  generally  more  difficult  but  important  case  of  distributed 
detection.  This  type  of  scheme  is  of  practical  significance  and  also  presents  interesting 
problems  of  analysis.  In  distributed  signal  detection  each  sensor  does  a  local  processing  of 
its  data  before  passing  on  the  result  to  a  central  processor.  The  central  processor  makes  the 
final  decision  based  on  the  results  it  gets  from  all  the  local  sensor  processors.  The  local 
processors  effect  a  data  reduction,  often  only  to  one-bit  local  decisions,  out  of 
considerations  of  communication  capacities,  security,  or  reliability.  The  design  of  the 
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system  for  optimum  overall  performance  requires  consideration  jointly  of  local  and  centra] 
processor  decision  rules,  and  can  get  quite  difficult.  Previous  analyses  have  generally 
assumed  that  the  observations  at  the  different  sensors  are  statistically  independent.  This  is 
clearly  not  always  reasonable;  for  example,  if  the  signal  is  emanating  from  a  single  source  it 
will  tend  to  be  correlated  at  each  of  the  sensors.  We  have  obtained  some  very  good  general 
results  on  locally  optimum  distributed  detection  schemes,  and  are  hoping  to  continue  our 
work  in  this  and  related  directions  during  the  next  grant  period.  The  dissertation  (21  was 
completed  recently  and  contains  some  significant  new  results  on  distributed  signal  detecnon 
in  non-Gaussian  noise.  It  describes  results  giving  explicitly  the  structures  of  the  local  and 
central  processors  for  dependent  observations  due  to  correlated  random  signal  arrivals. 

The  analysis  is  carried  out  for  weak  signals,  which  allows  asymptotic  techniques  to  be 
used.  The  results  are  interesting  generalizations  of  earlier  work  on  weak-signal-optimum 
centralized  detection  structures.  Some  preliminary  results  on  this  topic  were  published  in 
[21].  A  full  paper  on  this  aspect  of  signal  detection  theory  has  been  published  very  recently 
[11].  In  [2]  the  distributed  signal  detection  problem  was  also  viewed  as  a  problem  of 
optimum  quantization .  and  some  new  results  were  established  on  optimum  quantization  for 
weak  signal  detection. 

The  dissertation  [2]  also  contains  an  application  of  distributed  signal  detection  in 
constant  false  alarm  rate  iCFAR )  detection  for  radar  pulses  in  exponentially  distributed 
noise  and  clutter.  The  cell-averaging  CFAR  scheme  was  analyzed  for  two-sensor 
distributed  systems,  with  some  interesting  results  on  performance  improvements.  In  other 
work  on  CFAR  detection  that  has  been  accepted  for  journal  publication  [14],  we  have  made 
some  interesting  observations  on  the  statistical  optimality  (based  on  invariance,  sufficiency 
and  completeness)  of  the  classical  cell-averaging  CFAR  detector. 

Finally,  we  have  completed  work  on  one  aspect  of  nonparametric  detection  of 
random  signals  which  adds  to  the  existing  body  of  results  on  nonparametric  tests  in  the 
statistics  literature.  The  locally  most  powerful  rank  test  structure  has  been  derived  by  us 
for  random  signals  in  additive  random  noise,  a  problem  involving  convolutional 
combinations  of  signal  and  noise  pdf  s.  These  results  will  appear  shortly  in  the  journal 
paper  [13]. 

C.  Nonlinear  Radial  Basis  Function  Networks  in  Signal  Processing 

Lately  we  began  looking  into  the  use  of  nonlinear  networks  related  to  neural 
networks  but  which  have  a  simpler  structure  making  them  very  interesting  for  adaptive 
signal  processing  applications.  The  radial  basis  function  (RBF)  network  allows  the  output 
to  be  a  nonlinear  function  of  a  multivariate  input,  with  weights  or  parameters  which  appear 
in  a  linear  manner  in  the  output.  The  nonlinear  elements  are  radially  invariant  functions  of 
the  differences  between  the  input  and  a  set  of  "centers".  This  structure  turns  out  to  have 
some  appealing  characteristics,  and  in  preliminary  work  we  have  applied  it  to  accomplish 
adaptive  interference  cancellation  when  the  interference  appears  in  an  available  reference 
input  in  a  nonlinear  manner.  The  work  we  have  done  strongly  suggests  that  further  work 
is  warranted  to  exploit  the  unique  capability  that  this  type  of  processing  structure  seems  to 
allow.  In  [28]  we  give  our  initial  results  on  the  interference  cancelling  application,  and 
show  that  major  improvements  in  performance  can  be  realized  compared  to  linear  adaptive  * 

filter  structures. 
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D.  Other  Activity 


During  this  grant  period  the  Principal  Investigator  nas  completed  a  book  chapter  on 
advances  in  Nonparametric  Detection  15 1  which  will  appear  in  a  compendium  ot'  recent 
advances  in  the  field  of  signal  detection.  He  also  began  working  on  a  book  on  Statistical 
Signal  Processing, 

The  Principal  Investigator  has  given  invited  talks  at  several  universities  on  these 
research  areas,  and  has  also  made  an  invited  presentation  at  a  recent  workshop  held  in 
conjunction  with  the  Seventh  International  Conference  on  Multivariate  Analysis. 

He  served  as  Associate  Editor  for  Detection  Theory  for  the  IEEE  Transactions  on 
Information  Theory. 

The  Principal  Investigator  is  on  the  advisory  board  for  the  International  Conference 
on  Nonlinear  Signal  Processing  held  in  Finland  in  January  1993. 

He  was  also  the  research  advisor  of  another  full-time  graduate  student  (Rich 
Kozick)  at  the  University  of  Pennsylvania,  who  was  supported  on  a  separate  Air  Force 
Laboratory  Graduate  Fellowship. 
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